Efficient Elastic Net Regularization for Sparse Linear Models

This paper presents an algorithm for efficient training of sparse linear models with elastic net regularization. Extending previous work on delayed updates, the new algorithm applies stochastic gradient updates to non-zero features only, bringing weights current as needed with closed-form updates. Closed-form delayed updates for the $\ell_1$, $\ell_{\infty}$, and rarely used $\ell_2$ regularizers have been described previously. This paper provides closed-form updates for the popular squared norm $\ell^2_2$ and elastic net regularizers. We provide dynamic programming algorithms that perform each delayed update in constant time. The new $\ell^2_2$ and elastic net methods handle both fixed and varying learning rates, and both standard {stochastic gradient descent} (SGD) and {forward backward splitting (FoBoS)}. Experimental results show that on a bag-of-words dataset with $260,941$ features, but only $88$ nonzero features on average per training example, the dynamic programming method trains a logistic regression classifier with elastic net regularization over $2000$ times faster than otherwise

A functional spiking-neuron model of activity-silent working memory in humans based on calcium-mediated short-term synaptic plasticity

In this paper, we present a functional spiking-neuron model of human working memory (WM). This model combines neural firing for encoding of information with activity-silent maintenance. While it used to be widely assumed that information in WM is maintained through persistent recurrent activity, recent studies have shown that information can be maintained without persistent firing; instead, information can be stored in activity-silent states. A candidate mechanism underlying this type of storage is short-term synaptic plasticity (STSP), by which the strength of connections between neurons rapidly changes to encode new information. To demonstrate that STSP can lead to functional behavior, we integrated STSP by means of calcium-mediated synaptic facilitation in a large-scale spiking-neuron model and added a decision mechanism. The model was used to simulate a recent study that measured behavior and EEG activity of participants in three delayed-response tasks. In these tasks, one or two visual gratings had to be maintained in WM, and compared to subsequent probes. The original study demonstrated that WM contents and its priority status could be decoded from neural activity elicited by a task-irrelevant stimulus displayed during the activity-silent maintenance period. In support of our model, we show that it can perform these tasks, and that both its behavior as well as its neural representations are in agreement with the human data. We conclude that information in WM can be effectively maintained in activity-silent states by means of calcium-mediated STSP

Visual and auditory temporal integration in healthy younger and older adults

As people age, they tend to integrate successive visual stimuli over longer intervals than younger adults. It may be expected that temporal integration is affected similarly in other modalities, possibly due to general, age-related cognitive slowing of the brain. However, the previous literature does not provide convincing evidence that this is the case in audition. One hypothesis is that the primacy of time in audition attenuates the degree to which temporal integration in that modality extends over time as a function of age. We sought to settle this issue by comparing visual and auditory temporal integration in younger and older adults directly, achieved by minimizing task differences between modalities. Participants were presented with a visual or an auditory rapid serial presentation task, at 40-100 ms/item. In both tasks, two subsequent targets were to be identified. Critically, these could be perceptually integrated and reported by the participants as such, providing a direct measure of temporal integration. In both tasks, older participants integrated more than younger adults, especially when stimuli were presented across longer time intervals. This difference was more pronounced in vision and only marginally significant in audition. We conclude that temporal integration increases with age in both modalities, but that this change might be slightly less pronounced in audition

Use of Remote Monitoring to Improve Outcomes in Patients with Heart Failure: A Pilot Trial

Remote monitoring (RM) of homebound heart failure (HF) patients has previously been shown to reduce hospital admissions. We conducted a pilot trial of ambulatory, non-homebound patients recently hospitalized for HF to determine whether RM could be successfully implemented in the ambulatory setting. Eligible patients from Massachusetts General Hospital (n = 150) were randomized to a control group (n = 68) or to a group that was offered RM (n = 82). The participants transmitted vital signs data to a nurse who coordinated care with the physician over the course of the 6-month study. Participants in the RM program had a lower all-cause per person readmission rate (mean = 0.64, SD ± 0.87) compared to the usual care group (mean = 0.73, SD ± 1.51; P-value = .75) although the difference was not statistically significant. HF-related readmission rate was similarly reduced in participants. This pilot study demonstrates that RM can be successfully implemented in non-homebound HF patients and may reduce readmission rates

Identifying network communities with a high resolution

Community structure is an important property of complex networks. An automatic discovery of such structure is a fundamental task in many disciplines, including sociology, biology, engineering, and computer science. Recently, several community discovery algorithms have been proposed based on the optimization of a quantity called modularity (Q). However, the problem of modularity optimization is NP-hard, and the existing approaches often suffer from prohibitively long running time or poor quality. Furthermore, it has been recently pointed out that algorithms based on optimizing Q will have a resolution limit, i.e., communities below a certain scale may not be detected. In this research, we first propose an efficient heuristic algorithm, Qcut, which combines spectral graph partitioning and local search to optimize Q. Using both synthetic and real networks, we show that Qcut can find higher modularities and is more scalable than the existing algorithms. Furthermore, using Qcut as an essential component, we propose a recursive algorithm, HQcut, to solve the resolution limit problem. We show that HQcut can successfully detect communities at a much finer scale and with a higher accuracy than the existing algorithms. Finally, we apply Qcut and HQcut to study a protein-protein interaction network, and show that the combination of the two algorithms can reveal interesting biological results that may be otherwise undetectable.Comment: 14 pages, 5 figures. 1 supplemental file at http://cic.cs.wustl.edu/qcut/supplemental.pd

Inhibition in multiclass classification

The role of inhibition is investigated in a multiclass support vector machine formalism inspired by the brain structure of insects. The so-called mushroom bodies have a set of output neurons, or classification functions, that compete with each other to encode a particular input. Strongly active output neurons depress or inhibit the remaining outputs without knowing which is correct or incorrect. Accordingly, we propose to use a classification function that embodies unselective inhibition and train it in the large margin classifier framework. Inhibition leads to more robust classifiers in the sense that they perform better on larger areas of appropriate hyperparameters when assessed with leave-one-out strategies. We also show that the classifier with inhibition is a tight bound to probabilistic exponential models and is Bayes consistent for 3-class problems. These properties make this approach useful for data sets with a limited number of labeled examples. For larger data sets, there is no significant comparative advantage to other multiclass SVM approaches

Inhibition in multiclass classification

The role of inhibition is investigated in a multiclass support vector machine formalism inspired by the brain structure of insects. The so-called mushroom bodies have a set of output neurons, or classification functions, that compete with each other to encode a particular input. Strongly active output neurons depress or inhibit the remaining outputs without knowing which is correct or incorrect. Accordingly, we propose to use a classification function that embodies unselective inhibition and train it in the large margin classifier framework. Inhibition leads to more robust classifiers in the sense that they perform better on larger areas of appropriate hyperparameters when assessed with leave-one-out strategies. We also show that the classifier with inhibition is a tight bound to probabilistic exponential models and is Bayes consistent for 3-class problems. These properties make this approach useful for data sets with a limited number of labeled examples. For larger data sets, there is no significant comparative advantage to other multiclass SVM approaches

Lung cancer treatment costs, including patient responsibility, by disease stage and treatment modality, 1992 to 2003

AbstractObjectivesThe objective of this analysis was to estimate costs for lung cancer care and evaluate trends in the share of treatment costs that are the responsibility of Medicare beneficiaries.MethodsThe Surveillance, Epidemiology, and End Results (SEER)-Medicare data from 1991–2003 for 60,231 patients with lung cancer were used to estimate monthly and patient-liability costs for clinical phases of lung cancer (prediagnosis, staging, initial, continuing, and terminal), stratified by treatment, stage, and non-small- versus small-cell lung cancer. Lung cancer-attributable costs were estimated by subtracting each patient's own prediagnosis costs. Costs were estimated as the sum of Medicare reimbursements (payments from Medicare to the service provider), co-insurance reimbursements, and patient-liability costs (deductibles and “co-payments” that are the patient's responsibility). Costs and patient-liability costs were fit with regression models to compare trends by calendar year, adjusting for age at diagnosis.ResultsThe monthly treatment costs for a 72-year-old patient, diagnosed with lung cancer in 2000, in the first 6 months ranged from $2687 (no active treatment) to$9360 (chemo-radiotherapy); costs varied by stage at diagnosis and histologic type. Patient liability represented up to 21.6% of care costs and increased over the period 1992–2003 for most stage and treatment categories, even when care costs decreased or remained unchanged. The greatest monthly patient liability was incurred by chemo-radiotherapy patients, which ranged from $1617 to$2004 per month across cancer stages.ConclusionsCosts for lung cancer care are substantial, and Medicare is paying a smaller proportion of the total cost over time